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At least one drawing originally filed was informal and the print reproduced here is taken from a later filed formal copy. 
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.1. 2313216 

MULTIMEDIA INFORMATION MANAGEMENT SYSTEM 
) AND METHOD OF USING SAME 



Technical Field 

5 This invention relates in general to information management and 

more particularly to management of both real-time information and stored 
information. 



Background 

10 Considerable effort is being expended in order to design and integrate 

audio/video and computing technologies into single component equipment. 
This equipment may take the form of a television, radio and computer all 
combined into a single package. In view of the tremendous utility for such a 
device, this equipment is intended, for not only use with business 

15 applications, but also for home entertainment as well. 

Although these devices does have the capability to combine a number 
of media into a single component, it will only run each of these media 
applications separately. Each media is processed on a different system 
within the integrated device with little or no interaction. Thus, audio and 

20 video media will run on their respective portions of the device while any 

computing functions, such as access to the Internet or World Wide Web, are 
run on their respective portions of the integrated device. 

In some instances, specific media applications are capable of 
running simultaneously however no "real time" information that is 

25 received by a user is permitted to interact with those on stored or electronic 
media. For example, a viewer may be watching television however no 
Internet or computer applications are generally possible between these two 
media since they are independent and are not linked. Hence, although the 
television and computer systems may be operating simultaneously there is 

30 no interaction therebetween. This results in a great deal of related 
information, that would be helpful to the viewer, that cannot be easily 
accessed. 

Therefore, a system and method are needed for permitting 
interaction to effectively interconnect real-time media services like radio or 
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television with stored media services such as those accessible through the 
) use of the Internet or Compact Disk/ Read Only Memory (CD-ROM), 

Summary of the Invention 

5 A multimedia communication system for providing access between 

real-time multimedia information and stored multimedia information 
comprising; a receiver for receiving real-time multimedia information; at 
least one interface for accessing stored multimedia information; a 
processor for interpreting a predetermined portion of the real-time 

10 information from the first receiver such that the stored multimedia 
information is searched through the at least one interface to enable any 
corresponding stored information to be presented to a user. 

Brief Description of the Drawings 
15 FIG. 1 is a block diagram of the multimedia information 

management system according to the preferred embodiment of the 
invention. 

FIG. 2 is a flow chart showing the method of managing information 
received from a real-time information source and one or more stored media 
20 services according to the preferred method of the invention. 

Detailed Description of the Preferred Embodiment 

Referring now to FIG. 1, the multimedia information management 
system 100 includes a radio frequency (RF) receiver 101 and a cable interface 

25 103. The RF receiver 101 is used to receive real-time television and/or radio 
broadcasts through a radio frequency interface or via a physical link such 
as a cable television system or the like. 

The RF receiver 101 will typically be a standard television receiver or 
radio equipment capable of receiving any type of commonly broadcast 

30 programming using audio/video media. The cable interface 103 is used to 
transmit and receive data from any number of commercially accessible 
electronic storage media. These electronic media may include such data 
bases such as LEXIS/NEXIS™, DIALOG™ or the like. They will also 
include those resources that are accessible through the Internet or World 

35 Wide Web. Additionally, it will also be evident to those skilled in the art, the 



communications access cable 105, used to communicate this information, 
may also provide any other number of services such as telephone, video on- 
demand access, cable television and the like. The cable interface 103 can 
interface with a fiber optic cable, coaxial cable, a twisted copper pair or the 
like. 

Both the RF receiver 101 and the cable interface 103 are controlled 
using a microprocessor associated with a processor system 107. The 
processor system 107 is a microprocessor with a high processing speed and 
computing power to rapidly interpret and process incoming information 
from both the RF receiver 101 and cable interface 103. Included, preferably 
within the processor system 107, is one or more data buffers) 109. The data 
buffer 109 is a first-in first-out type (FIFO) memory that receives incoming 
information from the RF receiver 101, cable interface 103, a recorder 117 
(discussed hereinafter), and a CD-ROM 115. As is well known in the art, 
the information in the FIFO memory will be queued such that data that is 
first received will be the first to be purged when the data buffer 109 reaches 
it's full capacity. Thus, the fixed capacity of the data buffer 109 permits only 
a predetermined amount of data to be stored at one time. Once at capacity, 
new data that is received is entered into the data buffer 109 while the oldest 
data is expelled. 

Since the information received by the RF receiver 101, will generally 
be audio and video information, a speech-to-text converter 111 is provided to 
convert audio information received by the processor system 107 into a 
textual format. Once in textual format, the information can be easily input 
to the data buffer 109 where it can be more readily processed and/or 
manipulated for future use. Alternatively, information or data containing 
pointers to further information sources can also be transmitted with the 
broadcast to the RF receiver 101 or cable interface 103. In that event, no 
speech-to-text conversion will be necessary, and a user, viewing the speech 
from a man-to-machine interface (MMI) 119 can select any topical 
information in which the user desires additional information. 

Due to the gross amount of text that will be generated from the 
speech-to-text converter 111, a user preference memory 113 is provided to aid 
in matching user-selected information topics with incoming textual 
information. Thus, a user of the multimedia information management 



system 100 can preselect a number of topics that are of particular interest. 
These topics may include key words, phrases or the like. As text is entered 
into the data buffer 109, a comparison is made by the processor system 107 
between the preselected topics and information, that are stored in the user 

5 preference memory 113, with data currently in the data buffer 109. 

In the event a corresponding match is found, the matching 
information will be used to generate a query and then perform a search of 
selected electronically stored multimedia services. Thus, the corresponding 
information will be used by the processor system 107 to search any 

10 designated electronically stored information. These designated electronic 
storage media can be accessed through the cable interface 103 and include 
that information on the Internet or those stored locally such as on a CD- 
ROM 115. 

Moreover, to even further enhance the capability of the multimedia 

16 information management system 100, a audio/video recorder 117 is also 
connected to the processor system 107. The audio/video recorder 117 is 
typically a standard video cassette recorder (VCR) that can record both 
audio and video information from the RF receiver 101 or from the cable 
interface 103. The recorder 117 gives, a user of the multimedia information 

20 management system 100, the capability to record broadcast programming 
that can be replayed at a later time. At the time the recorded information is 
replayed in real time, the information is then processed using the speech-to- 
text converter 111 just as if the recorded information were currently being 
received in real time. As indicated above, any audio portions of recorded 

25 information are then converted to text, queued in the data buffer 109 and 
subsequently used to form a query. A search for related information can 
then be performed should the user desire. 

Finally in order to control the operation of the multimedia 
information management system 100, a man-to-machine interface (MMI) 

30 1 19 is provided. The MMI 1 19 allows a user the capability to fully control 
operating functions of the information management system 100. In the 
preferred embodiment of the invention, the user preferably views all 
information received at the RF receiver 101 and the cable interface 103 at the 
MMI 119. Thus, information is received at the RF receiver 101 and is 



displayed and interacts with other information at the MMI 119 where it can 
be selected and manipulated by the user. 

The MMI 119 includes a number of user options or functions 
including entering user preferences into the user preference memory 113 
and controlling the size of the information stored in the data buffer 109. 
Also, the MMI 119 is used for controlling information in the data buffer 109 
that is to be searched using one or more electronic storage media. 
Moreover, the MMI 119 allows the user to instantaneously select text 
information in the data buffer 109 while a broadcast is in progress. This 
situation may present itself during a news broadcast or whenever real time 
information is viewed at a man-to-machine interface (MMI) 119. 

In FIG. 2, the preferred method of providing a user interface between 
real-time (R/T) information and electronically stored information 200 is 
shown. Initially real-time information is received 201 by the RF receiver or 
cable interface where it may be recorded or played back 203 at a later time. 
Information that is then received by either the RF receiver or by playback 
though the use of a video cassette recorder is converted 205 to textual data 
using a speech-to-text converter. The textual data is then queued and stored 
207 into a data buffer where they can be selected for later processing. 

Segments of the text data are selected 209 where they are then 
displayed 208 to the user at the MMI and/or compared 211 with 
predetermined topic information which has been selected by the user. 
Based on related or corresponding information found during this 
comparison, a query is formulated 213 to determine if other electrically 
stored information is available on the same or similar topics. Additionally, 
a user will have the option of real time intervention using the MMI 119 to 
capture speech from the data buffer 109. The speech-to-text operation is 
then initiated at the processor system 107 using the MMI 119. 

Thereafter, a search is performed 215 for any related information on 
electronically stored media. These media can be accessed through the use 
of such means as the Internet or World Wide Web or through locally 
available electrically stored media such as a magnetic tape or CD-ROM. 
Any information retrieved dining the media search can then be presented 
217 to the user by a display on a television or the MMI 119. 
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While the preferred embodiments of the invention have been 
illustrated and described, it will be clear that the invention is not so limited. 
Numerous modifications, changes, variations, substitutions and 
equivalents will occur to those skilled in the art without departing from the 
5 spirit and scope of the present invention as defined by the appended claims. 

What is claimed is: 



Claims 

1. A multimedia communication system for providing access between real 
time multimedia information and stored multimedia information 
comprising: 

5 a receiver for receiving real-time multimedia information; 

at least one interface for accessing stored multimedia information; 

and 

a processor for interpreting a predetermined portion of the real-time 
multimedia information from the first receiver such that the stored 
10 multimedia information is searched through the at least one interface to 
enable any corresponding stored information to be presented to a user. 



2. A multimedia communication system as in claim 1 further comprising: 

a data buffer attached to the processor for selectively capturing a 
15 predetermined quantity of data from the receiver for interpretation by the 
processor. 

3. A multimedia communication system as in claim 1 wherein pointer data 
is received from the real-time multimedia information for accesssing the 

20 corresponding stored information. 

4. A multimedia communication system as in claim 2 wherein the data 
buffer is a first-in first-out (FIFO) memory. 

25 5. A multimedia communication system as in claim 1 further comprising: 
a user preference memory for indicating to the processor when 
predetermined topics of real-time information are being interpreted by the 
processor. 

30 6. A multimedia communication system as in claim 1 further comprising: 
a speech-to-text converter for converting audio information received by the 
receiver into textual information to be used by the processor. 

7. A multimedia communication system as in claim 1 wherein the at least 
35 one interface accesses locally stored and externally stored information. 
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8. A multimedia interface for use in providing interaction between 
substantially real-time multimedia information and electronically stored 
multimedia information comprising: 

5 a data buffer for storing selective portions of received real-time 

information; and 

at least one processor for utilizing the selective portions of real-time 
multimedia information to search for related information in the 
electronically stored multimedia information. 

10 

9. A multimedia interface as in claim 8 wherein the multimedia interface 
further includes: 

a user preference memory for storing predetermined user topics; 

and 

15 wherein the at least one processor compares the predetermined user 

topics with the selective portions of received real-time information for 
determining searchable information. 

10. A multimedia interface as in claim 8, wherein the at least one processor 
20 includes a user interface for selecting the real-time information to be 

searched and for entering the predetermined user topics. 

11. A multimedia interface as in claim 8, wherein the real-time 
information includes television broadcasting, radio broadcasting, or 

25 audio/video recorded media. 

12. A multimedia interface as in claim 8, wherein the at least one processor 
connects with an electronic media interface for accessing electronically 
stored information via a source connected by cable. 

30 

13. A method of providing a user interface between real-time information 
and electronically stored information comprising the steps of: 

receiving real-time information; 

storing predetermined portions of the received real-time information 
35 into a buffer; 



selecting segments of the stored real-time information; 
processing the selected segments to determine related information; 
searching at least one electronic storage media for the related 
information to obtain retrieved information; and 

presenting the retrieved information to a user. 

14. A method of providing a user interface as in claim 13 wherein the step 
of receiving father includes the step of: 

accessing the related information from the at least one electronic 
10 storage media using pointer data received directly from the real-time 
multimedia information. 

15. A method of providing a user interface as in claim 13, wherein the step 
of processing includes the step of: 

15 reading portions of stored real-time information from a first-in first- 

out memory; 

comparing the portions of stored real-time information with 
preselected user preference information from a user preference memory to 
determine the related information; and 
20 formulating a query based on the related information for submission 

to the at least one electronic storage media. 

16. A method of providing a user interface as in claim 13, wherein the step 
of receiving further includes the step of: 

25 converting audio information received in real-time to textual 

information. 

17. A method of providing a user interface as in claim 13, wherein the real- 
time information includes radio broadcasting, television broadcasting or 

30 audio/video recorded media. 
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18. A multimedia communications system substantially as hereinbefore 
described in FIG. 1 of the accompanying drawings. 
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19. A multimedia interface substantially as hereinbefore described with 
reference to the FIG. 1 in the accompanying drawings. 

20. A method substantially as hereinbefore described with reference to FIG. 
2 in the accompanying drawings. 
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